Search for: All records

Automatic differentiation (AutoDiff) in machine learning is largely restricted to expressions used for neural networks (NN), with the depth rarely exceeding a few tens of layers. Compared to NN, numerical simulations typically involve iterative algorithms like time steppers that lead to millions of iterations. Even for modest-sized models, this may yield infeasible memory requirements when applying the adjoint method, also called backpropagation, to time-dependent problems. In this situation, checkpointing algorithms provide a trade-off between recomputation and storage. This paper presents the package Checkpointing.jl that leverages expression transformations in the programming language Julia and the package ChainRules.jl to automatically and transparently transform loop iterations into differentiated loops. The user may choose between various checkpointing algorithm schemes and storage devices. We describe the unique design of Checkpointing.jl and demonstrate its features on an automatically differentiated MPI implementation of Burgers’ equation on the Polaris cluster at the Argonne Leadership Computing Facility. 

Abstract. In sequential estimation methods often used in oceanic and general climatecalculations of the state and of forecasts, observations act mathematicallyand statistically as source or sink terms in conservation equations for heat, salt, mass, and momentum.These artificial terms obscure the inference of the system's variability or secular changes.Furthermore, for the purposes of calculating changes inimportant functions of state variables such as total mass and energy orvolumetric current transports, results of both filter and smoother-based estimates are sensitive to misrepresentationof a large variety of parameters, including initial conditions, prioruncertainty covariances, and systematic and random errors in observations.Here, toy models of a coupled mass–spring oscillator system and of a barotropic Rossby wave system are used todemonstrate many of the issues that arise from such misrepresentations.Results from Kalman filter estimates and those from finite intervalsmoothing are analyzed.In the filter (and prediction) problem, entry of data leads to violation ofconservation and other invariant rules.A finite interval smoothing method restores the conservation rules, butuncertainties in all such estimation results remain. Convincing trend andother time-dependent determinations in “reanalysis-like” estimates require a full understanding of models, observations, and underlying error structures. Application of smoother-type methods that are designed for optimal reconstruction purposes alleviate some of the issues.